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ABSTRACT 



This study examined the conceptions of teacher evaluation 
criteria of 9th- and 12th-grade teachers and administrators. Four 
representative members of each group (n=12) were interviewed and asked to 
identify and define indicators of instructional practice based on the context 
of their allocated roles and experiences. It was hypothesized that 
conceptions of the criteria for teacher evaluation would vary according to 
the interviewed subjects' organizational positions as teachers or 
administrators. Variables were constructed from the responses of those 
interviewed and entered into a principal components analysis. Inspection of 
the scree plot and eigenvalues revealed three components that accounted for 
68% of the data. Ninth-grade teachers, twelfth-grade teachers, and 
administrators differed in their conceptions of teacher evaluation criteria. 
Ninth-grade teachers focused on management and discipline issues. 
Twelfth-grade teachers emphasized content material and student achievement. 
Administrators noted that teacher commitment and general instructional 
behaviors and skills were important attributes to consider for evaluation. 
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Abstract 



The proposed study examined ninth and twelfth grade 
teachers' and administrators' conceptions of teacher evaluation 
criteria. Four representative members of each group (N=12) were 
interviewed and asked to identify and define indicators of 
instructional practice based on the context of their allocated 
roles and experiences. It was hypothesized that conceptions' of 
the criteria for teacher evaluation would vary according to the 
interviewees' organizational positions (e.g., ninth or twelfth 
grade teachers, and administrators). Variables were constructed 
from the responses of the twelve interviewees and entered into a 
principal components analysis, inspection of the scree plot and 
eigenvalues revealed three components which accounted for 68 
percent of the data. Ninth grade teachers, twelfth grade 
teachers, and administrators differed on their conceptions of 
teacher evaluation criteria. Ninth grade teachers focused on 
management and discipline issues. Twelfth grade teachers 
emphasized content material, and student achievement. 
Administrators noted that teacher commitment and general 
instructional behaviors and skills were important attributes to 
consider for evaluation. 
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Introduction 

Although teacher evaluation criteria are most prevalently 
derived from research on effective teaching, to date there is no 
codified body of knowledge that theoretically, or empirically 
defines effective teaching (Haertel, 1990). Researchers and 
practitioners concur that most sets of criteria cannot adequately 
represent the features of competent performance across 
disciplines and across grade levels (Sykes, 1990). 

Based on the Theory of Evaluation and the Exercise of 
Authority (Dornbusch & Scott, 1975), conceptions of the criteria 
will vary according to reference group and organizational 
position. As an example, ninth and twelfth grade teachers serve 
student populations with distinct characteristics (early and late 
periods of adolescence). Administrators, on the other hand, have 
different organizational concerns and represent traditional roles 
as evaluators. 

There is a heightened recognition that teaching may be a 
phenomenon that cannot be adequately described without taking 
into account the views of those involved in the local educational 
organization (McLaughlin, 1990). The notion that evaluative 
criteria should be set and defined at the local level is 
consistent with the view of effective teaching as a multi- 
dimensional, value-bound and socially constructed endeavor (Good 
& Mulryan, 1990). A model which more closely approaches the 
reality of effective teaching would comprise an array of 
conceptions derived from the constituent sources within an 
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organization (McLaughlin, 1990). 

Researchers suggest that stakeholders in the process must 
have a common understanding of clearly articulated criteria 
(Joint Committee on Standards for Educational Personnel, 1988). 
Nevertheless, although 'Defined Role' guidelines offer notable 
suggestions for deriving and establishing relevant criteria, 
absent are guidelines for post facto means of determining whether 
the criteria, once established, are indeed relevant to the 
population, and whether evaluators and evaluatees have similar 
conceptions of the criteria. As clearly delineated as the 
criteria might be at the onset, without provision for measuring 
whether the conceptions of criteria are similar, differential 
interpretations and misconceptions are not only likely, but are 
to be anticipated. 

Externally imposed criteria of teacher evaluation may not 
reflect the views of the constituents within a particular school 
(Haertel, 1990). Individuals' experiences within a specific 
school context influence their views of what constitutes "good” 
teaching. These experiences affect individuals' conceptions of 
valid indicators of teacher performance. Rather than using 
externally derived criteria, alternative methods that rely on 
individuals' experiences and conceptions within a specific school 
may generate more valid indicators of teacher evaluation. 

The purpose of this study was to investigate whether 
conceptions of teacher evaluation criteria varied across 
different organizational positions (ninth grade teachers, twelfth 
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grade teachers, and administrators). Individual representatives 
from each of the three organizational groups were interviewed; 
through these interviews, conceptions were gathered and compared 
within and across organizational positions. These individuals 
provided descriptions of what they perceived to be fundamental 
teacher evaluation criteria. Each group (ninth grade teachers, 
twelfth grade teachers, and administrators) serves different 
populations, which generates different experiences across the 
three groups; these unique experiences were expected to influence 
divergent perceptions of teacher evaluation criteria across the 
three groups. 

This study addressed the following questions: 

1) What are the criteria deemed important by constituent groups 
within one secondary school? 

2) Are there differences across administrators, ninth grade 
teachers, and twelfth grade teachers in the conceptions of 
teacher evaluation criteria? 

Methods 

Sample 

The sample consisted of 12 educators in one rural secondary 
school in upstate New York. These 12 educators were comprised of 
three different groups: ninth grade teachers, twelfth grade 
teachers, and administrators. Within each grade level, teachers 
from different content areas were interviewed. Administrators 
included one principal, one recently retired principal, one 
assistant superintendent, and the dean of students. 
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Interviews 

Focus group and individual interviews were conducted by one 
researcher. Focus groups consisted of ninth and twelfth grade 
teachers. The intention of these focus group interviews was to 
obtain a broad range of ideas pertaining to teacher evaluation 
criteria. Individual interviews were conducted with four ninth 
grade and four twelfth grade teachers. These eight teachers had 
also been interviewed in the focus groups. The in-depth 
interviews were conducted to gather more specific information. 
Administrators were interviewed individually, and did not 
participate in the focus group interviews. 

Data Analysis 

Interviews were tape recorded and transcribed verbatim. 

These data were entered into QSR*NUDIST, a qualitative data 
management program. Two coders examined the data, categorized the 
individual responses, and assigned codes for each category. All 
individual responses were subsequently coded according to the 
specified categories, and the two raters established consensus. 
The categories were then collapsed according to similar themes. 

In order to address whether ninth grade teachers, twelfth 
grade teachers, and administrators differ as to their beliefs 
regarding the criteria that should be used to evaluate teacher 
performance, we created three dummy variables. For each of the 
three variables (ninth grade teachers, twelfth grade teachers, 
administrators), we coded one of the groups as "1" and all the 
rest as "0" . 
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The data were entered into principal components analysis 
with variraax rotation. Recently, Pruzek and Lepak (1992) 
demonstrated the plausibility of using principal components 
analysis in an exploratory fashion when conducting multivariate 
research with small samples. In addition, Guadagnoli and Velicer 
(1988) concluded that components with four or more loadings above 
.60 can be viewed as reliable. They also reported that for a wide 
range of sample sizes, results meeting that condition may be 
considered generalizable . A review of Table 2 shows that our 
results meet the conditions stated by Guadagnoli and Velicer. 

Results 

Results of an initial analysis with all variables included 
showed that three of the variables had extremely low loadings and 
communal it ies . While these variables were considered to be 
theoretically important, we chose to eliminate them, because they 
could not be considered to be reliable. Those variables included 
statements that referred to; 1) types of accommodations teachers 
make for students; 2) whether or not teachers consciously attempt 
to engage all students regardless of gender or "at-risk" status; 
and 3) whether the teachers' concept is that of facilitator or 
instructor. Thus, the analysis consisted of 14 variables (See 
Table 1 ) . 
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The remaining 14 variable were entered into a principal 
components analysis, inspection of the scree plot and eigenvalues 
revealed three components which accounted for 68 percent of the 
data. Table 2 summarizes the varimax rotated loadings of each 
variable on the three components. Variables with a loading of .50 
or higher were used to interpret a component. Each component is 
described below. 

Ninth Grade Teachers Management and Discipline Orientation 

The following six variables defined the first componen«t: 
specific management skills, general management skills, observable 
management techniques, teacher communicates goals and 
expectations, specific ways of implementing discipline, and 
policies and procedures related to discipline issues. The results 
indicate that ninth grade teachers were likely to consider these 
variables to be important as evaluation criteria, in contrast, 
the negative direction of the loading for twelfth grade teachers 
indicates that they would not view these variables as necessary 
evaluation criteria. 

Twelfth Grade Teachers' Student-Focused Skills. Content, and 
Achievement Orientation 

Loading on the second component were the following 
variables: student's achievement level and methods of assessment, 
student comfort level and rapport with teacher, teachers' use of 
materials and media, teachers' use of different types of 
questions, teachers' knowledge of subject matter, and teachers' 
general instructional behaviors and skills. Twelfth grade 
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teachers, the only group clearly identified with this component, 
would tend to emphasize these variables as important evaluation 
criteria. 

Administrators' Teacher-Focused Skills and Commitment 
This third component included variables which 
administrators were most likely to consider as important 
evaluation criteria. These variables included teacher's general 
instructional behaviors and skills, evidence of teacher 
commitment to the school and students, and level of teacher 
satisfaction and comfort. Ninth grade teachers were not likely to 
consider these variables to be of concern as evaluation criteria. 
Interestingly, administrators do not consider policy and 
procedures related to discipline issues as important evaluation 
criteria, while ninth grade teachers do consider these issues 
important . 
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Discussion 



According to the data, ninth grade teachers, twelfth grade 
teachers, and administrators have different conceptions of which 
criteria are important indicators of teacher performance. 

Teachers perceive students in ninth and twelfth grades 
differently, and thus their instructional emphasis will differ 
across these two populations. Administrators, on the other hand, 

want assurance and concrete evidence that teachers are adequately 

1 

performing their roles. These different groups deal with 
different populations, and therefore their experiences influence 
their instructional focus. 

Ninth Grade Teachers 

Ninth grade teachers in this study indicated that structural 
components of their teaching were the most important indicators 
of effective teaching performance. The six components identified 
comprised management, structure, and discipline issues. 

Ninth grade students are in early adolescence, and their 
developmental level may necessitate that teachers use a 
consistent, organized classroom approach. Early adolescents who 
enter ninth grade experience the "top-dog" phenomenon, which 
means that they have changed from the top position in middle 
school to the least powerful position in high school (Santrock, 
1996). These transitions which occur in early adolescence may be 
more disruptive than changes that occur either earlier or later 
in adolescence (Blyth, Simmons, & Carlton-Ford, 

1983) . 
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Piaget (1972) stated that adolescents are in the formal 
operational thought stage; they begin to think abstractly, 
ideally, and hypothetically. With the onset of formal operational 
thought, early adolescents begin to think idealistically, and 
they may overtly demonstrate their dissatisfaction with issues 
and individuals who do not measure up to their ideals. 

Another characteristic of adolescence is egocentrism, which 
refers to an adolescent's feeling of uniqueness; adolescents feel 
that others are as interested in them as they are themselves 
(Elkind, 1978). Early adolescents are particularly self-conscious 
about others' perceptions, and this egocentrism contributes to 
the display of surface behaviors (Santrock, 1996). Teachers who 
deal with young adolescents, therefore, may feel the need to 
focus on controlling the display of overt behaviors. Teachers who 
focus on structure may help them achieve balance and develop 
consistency in thought. 

Teachers have the potential to influence the social and 
cognitive development of early adolescents. This was demonstrated 
by statements made by ninth grade educators about the important 
elements of teaching students at that level. Erikson discussed 
the potential impact adults have on adolescents: 

"The strengths a young person finds in adults at this time- their 
willingness to let him experiment, their eagerness to confirm him 
at his best, their consistency in correcting his excesses, and 
the guidance they give him- will codetermine whether or not he 
eventually makes order out of necessary inner confusion and 
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applies himself to the correction of disordered conditions. He 
needs freedom to choose, but not so much freedom that he cannot, 
in fact, make a choice (Erikson, I960)." 

Twelfth Grade Teachers 

As opposed to students in ninth grade, by twelfth grade many 
of these issues have been resolved. Early adolescents use 
abstract reasoning to think ideally about social issues, thus 
creating idealistic rebellion (Rice, 1996). By late adolescence, 
however, students have had ample experiences to compare to .their 
ideals, and adolescents become more tolerant and understanding. 
Egocentrism which is present in early adolescence gradually 
diminishes (Elkind, 1978)). Teachers can then become more 
involved in curriculum, rather than focusing on the development 
of the students (Cowan, 1978). This is consistent with this 
study's findings. Twelfth grade teachers expressed more concern 
for subject matter, student rapport, presentation techniques, and 
analytical assessments that required more abstract thought. These 
teachers were no longer predominantly concerned with students' 
unpredictability and capacity for misinterpretation. 
Administrators 

In contrast to teachers, administrators have a different 
perspective of what criteria demonstrate effective teaching. 
Administrators identified indicators that reflected teachers' 
behaviors, rather than student-focused behaviors. The three 
criteria found to be important to administrators included general 
instructional teacher behaviors and skills, level of teacher 




17 



13 



satisfaction and comfort, and teacher's commitment to the school 
and students. These three indicators are observable criteria, and 
may reflect the administrator's role of ensuring that teachers 
overtly demonstrate behaviors that can be identified by 
observers. Administrators did not consider policy and procedures 
related to discipline as pertinent evaluation criteria. One 
plausible explanation is that administrators may not regard 
issues related to discipline as a central component of a 
teacher's role; they may consider discipline as an administrative 
responsibility . 

Implications 

Because this study was conducted in one school, replications 
should be undertaken to determine if representative groups in 
other schools have similar conceptions of teacher evaluation 
criteria, in addition, other grade levels (e.g., tenth grade 
teachers, eleventh grade teachers) should be studied for purposes 
of comparison. Further studies could also investigate schools 
with diverging theoretical orientations (e.g., magnet schools, 
humanistic schools, residential schools, etc.) to determine if 
conceptions of the criteria are influenced by the theoretical 
orientation of the school. 

When evaluating teachers, the grade level of the students 
should not be overlooked. According to this study, emphases will 
vary according to the grade level of the students. Therefore, 
criteria valued as important by the evaluator may not be judged 
important to the evaluatee. The two important actors in the 
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process may be operating according to two disparate standards. 
HOW does one measure a construct when the participants do not 
agree on the definition of the construct? 
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